Add fp8_e3m4 dtype support - #97
Conversation
Code ReviewSingle-pass review, ranked most-severe first. 1.
|
Fixes:
|
Code review findingsRanked most severe first. 1. Given tensor A of dtype e4m3 with value 100.0 (representable — e4m3 max is 448) and tensor B of dtype e3m4 with value 0.5, a ufunc calling 2. The diff comments out 3. Same shape as finding 2: fp16 (mantissa=10) and bf16 both strictly dominate e3m4 (mantissa=4, exp range narrower), so the widening is exact, but the table reports it as lossy. Every finite e3m4 value round-trips through float16 exactly. 4. Pattern arm Every other Dtype constructor in this match is written with a dot ( 5. Duplicate/misleading Line 442 is a verbatim copy of line 437 ( 6. Misleading comment on the e3m4/e4m3 join arm — The comment "e3m4 absorbs e4m3 (since e3m4 has 4 mantissa bits vs e4m3's 3)" justifies the (incorrect) absorption by mantissa width alone, ignoring that e4m3's exponent range (up to 448) is far wider than e3m4's (up to 15.5). A future maintainer who applies this reasoning to another pair may reproduce the range-truncation bug from finding 1. 7. Every other fp format uses Findings 1-3 are the correctness bugs; 4 is a latent hazard; 5-7 are cleanup. |
Fixes:
|
|
I read the full diff, then the enclosing functions and adjacent fp8 (e4m3/e5m2) paths in Finding 1 — e3m4 and e4m3 both serialize to Failure scenario: user creates a Tensor with dtype Finding 2 — The comment about e3m4->e4m3 lossiness is placed inside the e3m4-lossless list, between the The block reads: A future maintainer will parse the comment as annotating the Finding 3 — The new comment reads Finding 4 — The |
Fixes:
|
Code review findings (single-pass, high-effort)Note: single-pass review, Agent tool unavailable — no multi-agent fan-out or subagent verify pass ran. Ranked most severe first. 1. 2. 3. 4. 5. No further correctness bugs survived a re-check against the diff. The e3m4 encode/decode routines mirror the e5m2 pattern that already has a PBT-verified round-trip, and the new e3m4 round-trip PBT is included. The |
Fixes:
|
|
Single-pass code review (no Agent-tool fan-out). Findings: 1. 2. 3. 4. No correctness bugs found in the fp8_e3m4 encode/decode math itself — subnormal, normal, overflow-to-inf, NaN, and rounding boundaries all match the parallel e5m2 code and pass the ml_dtypes-verified |
Fixes:
|
Code review findings (single-pass, no Agent tool)Single-pass inline review of the 1. 2. 3. 4. 5. 6. 7. |
|
Reviewed this with a bit-exactness focus. First, the good news: the numeric core is exhaustively correct. I built the library clean (434 jobs, 0 errors / 0 warnings, so match exhaustiveness, the There's exactly one genuine numeric divergence, plus some serialization/API/test items worth a look. Should fix before merge1. NaN encodes to the wrong byte —
Suggested fix: use #guard (Float32.ofBits 0x7FC00000).toFloat8E3M4Bits == (0x78 : UInt8)There's no encode Highest-impact limitation (partly inherent — a design call)2. npy
API / design3.
Test coverage
Documented divergences (intentional — just flagging)
Minor cleanups
Overall — nice work; the hard part (the bit math) is correct. #1 is the one I'd fix before merge. |
…, and fix comments
Fixes:
Test Coverage:
Doc divergence:Updated the double-rounding comment to reflect that the divergence applies at interior values, not just the overflow edge. join(e3m4, e4m3) = none is intentional; numpy promotes to e3m4 which silently overflows e4m3 values above 15.5. Code cleanup:
|
Findings1. Dead guard in 2. Typos in Npy.lean comment (TensorLib/Npy.lean:134) 3. Misleading rationale in 4. Dead conditionals in e3m4 subnormal encode branch (TensorLib/Float.lean:604) What I did not findNo correctness bugs in the encode/decode math: exponent rebias, mantissa shift widths (19 for e3m4 vs. 20/21 for e4m3/e5m2), subnormal scale factor (2^-6), overflow-to-inf boundary (realExp > 3), and RTNE round bit / sticky masks all check out against the E3M4 (1s + 3e + 4m, bias 3) spec. The exhaustive 256-value round-trip PBT and the arithmetic tests independently corroborate this. |
Fixes:
|
Re-review resultsFix commit
Fresh pass over the whole |
fp8_e3m4: